{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Oracle Cloud Infrastructure Generative AI"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Oracle Cloud Infrastructure (OCI) Generative AI is a fully managed service that provides a set of state-of-the-art, customizable large language models (LLMs) that cover a wide range of use cases, and which is available through a single API.\n",
    "Using the OCI Generative AI service you can access ready-to-use pretrained models, or create and host your own fine-tuned custom models based on your own data on dedicated AI clusters. Detailed documentation of the service and API is available __[here](https://docs.oracle.com/en-us/iaas/Content/generative-ai/home.htm)__ and __[here](https://docs.oracle.com/en-us/iaas/api/#/en/generative-ai/20231130/)__.\n",
    "\n",
    "This notebook explains how to use OCI's Generative AI complete models with LangChain."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup\n",
    "Ensure that the oci sdk and the langchain-community package are installed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install -U oci langchain-community"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Usage"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.llms.oci_generative_ai import OCIGenAI\n",
    "\n",
    "llm = OCIGenAI(\n",
    "    model_id=\"cohere.command\",\n",
    "    service_endpoint=\"https://inference.generativeai.us-chicago-1.oci.oraclecloud.com\",\n",
    "    compartment_id=\"MY_OCID\",\n",
    "    model_kwargs={\"temperature\": 0, \"max_tokens\": 500},\n",
    ")\n",
    "\n",
    "response = llm.invoke(\"Tell me one fact about earth\", temperature=0.7)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Chaining with prompt templates"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.prompts import PromptTemplate\n",
    "\n",
    "llm = OCIGenAI(\n",
    "    model_id=\"cohere.command\",\n",
    "    service_endpoint=\"https://inference.generativeai.us-chicago-1.oci.oraclecloud.com\",\n",
    "    compartment_id=\"MY_OCID\",\n",
    "    model_kwargs={\"temperature\": 0, \"max_tokens\": 500},\n",
    ")\n",
    "\n",
    "prompt = PromptTemplate(input_variables=[\"query\"], template=\"{query}\")\n",
    "llm_chain = prompt | llm\n",
    "\n",
    "response = llm_chain.invoke(\"what is the capital of france?\")\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Streaming"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "llm = OCIGenAI(\n",
    "    model_id=\"cohere.command\",\n",
    "    service_endpoint=\"https://inference.generativeai.us-chicago-1.oci.oraclecloud.com\",\n",
    "    compartment_id=\"MY_OCID\",\n",
    "    model_kwargs={\"temperature\": 0, \"max_tokens\": 500},\n",
    ")\n",
    "\n",
    "for chunk in llm.stream(\"Write me a song about sparkling water.\"):\n",
    "    print(chunk, end=\"\", flush=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Authentication\n",
    "The authentication methods supported for LlamaIndex are equivalent to those used with other OCI services and follow the __[standard SDK authentication](https://docs.oracle.com/en-us/iaas/Content/API/Concepts/sdk_authentication_methods.htm)__ methods, specifically API Key, session token, instance principal, and resource principal.\n",
    "\n",
    "API key is the default authentication method used in the examples above. The following example demonstrates how to use a different authentication method (session token)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "llm = OCIGenAI(\n",
    "    model_id=\"cohere.command\",\n",
    "    service_endpoint=\"https://inference.generativeai.us-chicago-1.oci.oraclecloud.com\",\n",
    "    compartment_id=\"MY_OCID\",\n",
    "    auth_type=\"SECURITY_TOKEN\",\n",
    "    auth_profile=\"MY_PROFILE\",  # replace with your profile name\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dedicated AI Cluster\n",
    "To access models hosted in a dedicated AI cluster __[create an endpoint](https://docs.oracle.com/en-us/iaas/api/#/en/generative-ai-inference/20231130/)__ whose assigned OCID (currently prefixed by ‘ocid1.generativeaiendpoint.oc1.us-chicago-1’) is used as your model ID.\n",
    "\n",
    "When accessing models hosted in a dedicated AI cluster you will need to initialize the OCIGenAI interface with two extra required params (\"provider\" and \"context_size\")."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "llm = OCIGenAI(\n",
    "    model_id=\"ocid1.generativeaiendpoint.oc1.us-chicago-1....\",\n",
    "    service_endpoint=\"https://inference.generativeai.us-chicago-1.oci.oraclecloud.com\",\n",
    "    compartment_id=\"DEDICATED_COMPARTMENT_OCID\",\n",
    "    auth_profile=\"MY_PROFILE\",  # replace with your profile name,\n",
    "    provider=\"MODEL_PROVIDER\",  # e.g., \"cohere\" or \"meta\"\n",
    "    context_size=\"MODEL_CONTEXT_SIZE\",  # e.g., 128000\n",
    ")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "oci_langchain",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
